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STATUS OF CLAIMS 

Claims 1-24 are presently pending in the application and stand finally rejected. The 
rejections of claims 1-24 are being appealed. 

STATUS OF AMENDMENTS 

The amendment after final, filed December 26, 2006 was entered. The Examiner agreed in 
the Advisory Action mailed January 10, 2007 that this addressed the § 1 12 rejection. 

SUMMARY OF THE CLAIMED SUBJECT MATTER 

This invention relates to the field of image-based vehicle occupant detection, 
classification, and posture estimation. More specifically, the invention uses an imaging system 
in order to simultaneously monitor and classify all vehicle seating areas into a number of 
occupancy classes, the minimum of which includes (i) empty, (ii) occupied by an in-position 
adult, (iii) occupied by an out-of-position occupant, (iv) occupied by a child passenger, (v) 
occupied by a forward facing infant seat, (vi) occupied by a rear facing infant seat. 

Various solutions have been proposed to allow the modification of an airbag's 
deployment when a child or infant is occupying the front passenger seat. This could result in an 
airbag being deployed at a reduced speed, in an alternate direction, or not at all. The most basic 
airbag control systems include the use of a manual activation/deactivation switch controllable by 
the driver. Due to the nature of this device, proper usage could be cumbersome for the driver, 
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especially on trips involving multiple stops. Weight sensors have also been proposed as a means 
of classifying occupants, but have difficulty with an occupant moving around in the seat, an 
over-cinched seat belt on an infant seat, and can misclassify heavy but inanimate objects. 
Capacitance-based sensors have also been proposed for occupant detection, but can have 
difficulty in the presence of seat dampness. 

Vision-based systems offer an alternative to weight-based and capacitance-based 
occupant detection systems. Intuitively we know that vision-based systems should be capable of 
detecting and classifying occupants, since humans can easily accomplish this task using visual 
senses alone. A number of vision-based occupant detection/classification systems have been 
proposed. In each of these systems one or more cameras are placed within the vehicle interior 
and capture images of the front passenger seating seat region. The seat region is then observed 
and the image is classified into one of several pre-defined classes such as "empty," "occupied," 
or "infant seat." This occupancy classification can then act as an input to the airbag control 
system. 

Prior art image-based classification systems rely on a training set require that the 
classifier function be retrained if the camera mount location is moved, or used in a different 
vehicle. Finally, each of these systems is limited to observing a single seating area. Monitoring 
of multiple seating areas would require multiple devices to be installed, each focused on a 
different seating area. 
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This invention proposes an alternative in which all seating areas can be monitored from a 
single camera device. This invention is a vision-based device for use as a vehicle occupant 
detection/classification and posture estimation system. The end uses of such a device include 
acting as an input to an airbag control unit and dynamic airbag suppression. 

A wide-angle ("fish eye") lens equipped camera is mounted in the vehicle headliner such 
that it can capture images of all seating areas in the vehicle simultaneously. Image processing 
algorithms can be applied to the image to account for lighting, motion, and other phenomena. A 
spatial-feature vector is then generated which numerically describes the content of each seating 
area. This descriptor is the result of a number of digital filters being run against a set of sub- 
images, derived from pre-defined window regions in the original image. This spatial-feature 
vector is then used as an input to an expert classifier function, which classifies the seating area as 
best representing a scenario in which the seat is (i) empty, (ii) occupied by an adult, (iii) 
occupied by a child, (iv) occupied by a rear-facing infant seat (RFIS), (v) occupied by a front- 
facing infant seat (FFIS), or (vi) occupied by an undetermined object. When an occupant is 
determined to be in a seating area, the posture is estimated by further classifying them as (i) in 
position, or (ii) out-of -position and within the "keep out zone" of the airbag. When an occupant 
is within the "keep out zone," the airbag is dynamically suppressed to ensure the deployment 
does not injure an occupant who is positioned close to the deployment site. This expert classifier 
function is trained using an extensive sample set of images representative of each occupancy 
classification. Even if this classifier function has not encountered a similar scene through the 
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course of its training period, it will classify each seating area in the captured image based on 
which occupancy class generated the most similar filter response. Each seating area's occupancy 
classification from the captured image is then smoothed with occupancy classifications from the 
recent past to determine a best-estimate occupancy state for the seating area. This occupancy 
state is then used as the input to an airbag controller rules function, which gives the airbag 
system deployment parameters, based on the seat occupancy determined by the system. 

This invention makes no assumptions of a known background model and makes no 
assumptions regarding the posture or orientation of an occupant. The device is considered to be 
adaptive as once the expert classifier function is trained on one vehicle, the system can be used in 
any other vehicle by taking vehicle measurements and adjusting the system parameters of the 
device. 

Claim 1 

Referring to Figure 3 of the present application, independent claim 1 recites a method for 
classifying an occupant 24a-d, including the steps of capturing an image of a plurality of 
occupant seating areas 26a-d in a vehicle 22 (Page 7, lines 16-17; Figure 3) and dividing the 
image into a plurality of subimages of predetermined spatial regions. (Page 8, lines 4-6; Figure 
3). Claim 1 also recites the steps of generating a spatial feature matrix 47 of the image based 
upon the plurality of subimages (Page 9, lines 14-17; Figure 3), and analyzing the spatial feature 
matrix 47 (Page 12, line 12-13). The plurality of occupants 24a-d in the occupant seating areas 
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26a-d are then classified based upon the analysis of the spatial feature matrix 47. (Page 12, line 
14 to Page 14, line 7; Figure 3). 

Claim 20 

Referring to Figure 3 of the present application, independent claim 20 is another method 
for classifying an occupant 24a-d including the steps of capturing an image of a plurality of 
occupant areas 26a-d (Page 7, lines 16-17; Figure 3) and dividing the image into a plurality of 
subimages of predetermined spatial regions (Page 8, lines 4-6; Figure 3). A plurality of low- 
level descriptors arc generated from each of the plurality of subimages (Page 8, lines 8-11 and 
lines 14-22; Figure 3). The low-level descriptors are then analyzed and used to classify an 
occupant 24a-d in each of the plurality of occupant areas 26a-d. (Page 9, lines 16-23; Page 12, 
line 14 to Page 14, line 7; Figure 3). 

Claim 10 

Independent claim 10 recites a vehicle occupant classification system 20 including an 
image sensor 28 for capturing an image of a plurality of occupant areas 26a-d (Page 7, lines 16- 
18; Figure 1). The system further includes a processor 30 dividing the image into a plurality of 
subimages, the processor 30 analyzing the subimages to determine a classification of the 
occupants 24a-d in each of the plurality of occupant areas 26a-d. (Page 9, lines 16-23; Page 12, 
line 14 to Page 14, line 7; Figures 1 and 3). 
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GROUND OF REJECTION TO BE REVIEWED ON APPEAL 

Appellant seeks review of the following ground of rejection: 

Claims 1-24 have been rejected as anticipated by Kamei (U.S. Patent No. 5,528,698) 
ARGUMENTS 

Claims 1-24 

All of the claims have been rejected as anticipated by Kamei (U.S. Patent No. 5,528,698). 
The Examiner maintains his position that Kamei discloses "capturing an image of a plurality of 
occupant seating areas" and "classifying a plurality of occupants in the occupant seating areas." 
Kamei only discloses classification of a single occupant area - specifically the front passenger seat 
for the purpose of disabling the airbag. 

The Examiner takes several specific citations to Kamei out of context. If one were to read 
only the Examiner's argument and those specific citations, one might conclude that the rest of the 
patent must describe "capturing an image of a plurality of occupant seating areas." However, the 
rest of the document indicates that Kamei was only concerned with the front passenger seat. First, 
Kamei's abstract states, "The objects in the field of view are then discriminated to determine 
whether a rear-facing child carrier is located in the passenger seat, such that the passenger-side 
airbag can be disabled." Kamei's Description of Prior Art describes only the problems associated 
with rear-facing infant seats being placed in the front passenger seat in front of an airbag. 
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Additionally, Figures 1 and 2 show only the passenger seat in front of the airbag being analyzed and 
no other seats are shown being analyzed. Most conclusively, the flowchart, Figure 8, starts with the 
only three possibilities: 810 - Front passenger seat empty; 812 Child in rear-facing child-carrier 
seat in front passenger seat; or 814 - Other. Kamei explains, "In general, if scenarios 810 and 812 
are observed, the passenger side airbag will be disabled. In any other expected case 814, the airbag 
would be enabled." Thus, clearly, only the front passenger seat is being considered in all of the 
scenarios contemplated by Kamei. 

Reading the citations selected by the Examiner in context, one understands that Kamei 's 
occasional reference to "occupants" (plural) probably refers to occupants who might sit (one at a 
time) in the front passenger seat. Kamei does not disclose analyzing any seat other than the front 
passenger seat. 

As shown above, Kamei does not disclose "capturing an image of a plurality of occupant 
seating areas" or "classifying a plurality of occupants in the occupant seating areas." Therefore, 
none of claims 1-24 are anticipated by Kamei. 

Claims 1 and 20 

Additionally, Kamei does not divide the image into a plurality of subimages of 
predetermined spatial regions. There is no indication that the "sub image space 620" is a plurality of 
images divided from the original image. The "segmentor 622" referenced by the Examiner does not 
divide the image into a plurality of subimages, or if it does divide it in some interim step, the output 
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of the segmentor is an "enhanced image" (singular). This "enhanced image" is the input to the 
interest point locator 632 (col. 6, lines 27-30). Since the output is a single "enhanced image," 
Kamei could not possibly "generate a spatial feature matrix of the image based upon the plurality of 
subimages," (Claim 1). Similarly, with a single "enhanced image," Kamei could not "generate a 
plurality of low-level descriptors from each of the plurality of subimages" as recited in claim 20. 
For these additional reasons, Kamei does not anticipate claims 1 and 20. 

Claim 9 

Claim 9 depends from claim 1 and further recites, "the plurality of subimages overlap one 
another." Nothing in the specific citations by the Examiner, including references to Figures 1-3, 
indicates that Kamei discloses any overlapping subimages. Claim 9 is independently allowable. 

CLOSING 

For the reasons set forth above, the final rejection of all claims is improper and must be 
reversed. 

Respectfully submitted, 

/John E. Carlson/ 

John E. Carlson, Reg. No. 37,794 
Carlson, Gaskey & Olds PC 
400 W. Maple Rd., Suite 350 
Birmingham, MI 48009 
Dated: July 25, 2007 (248)988-8681 
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CLAIMS APPENDIX 

1 . A method for classifying an occupant including the steps of: 

a) capturing an image of a plurality of occupant seating areas in a vehicle; 

b) dividing the image into a plurality of subimages of predetermined spatial regions; 

c) generating a spatial feature matrix of the image based upon the plurality of subimages; 

d) analyzing the spatial feature matrix; and 

e) classifying a plurality of occupants in the occupant seating areas based upon said step 
d). 

2. The method of claim 1 further including the step of processing the image to 
account for lighting and motion before said step d). 

3. The method of claim 1 further including the step of smoothing the classification 
of the occupant over time. 

4. The method of claim 1 further including the step of determining whether to 
activate an active restraint based upon the classification of said step e). 

5. The method of claim 1 wherein said step d) further includes the step of applying 
expert classifier algorithm to the spatial feature matrix. 

6. The method of claim 5 wherein said step d) further includes the step of analyzing 
the spatial feature matrix based upon a set of training data. 
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7. The method of claim 6 further including the step of creating the set of training 
data by capturing a plurality of images of known occupant classifications of the occupant area. 

8. The method of claim 5 wherein the expert classifier algorithm includes a neural 
network. 

9. The method of claim 1 wherein the plurality of subimages overlap one another. 

10. A vehicle occupant classification system comprising: 

an image sensor for capturing an image of a plurality of occupant areas; and 

a processor dividing the image into a plurality of subimages, the processor analyzing the 
subimages to determine a classification of the occupants in each of the plurality of occupant 
areas. 

11. The vehicle occupant classification system of claim 10 wherein the processor 
determines the classification of the occupant from among the classifications including: adult, 
child and infant seat. 

12. The vehicle occupant classification system of claim 11 wherein the processor 
determines the classification of the occupant from among the classifications including: adult, 
child, forward-facing infant seat and rearward-facing infant seat. 
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13. The vehicle occupant classification system of claim 10 wherein the processor 
generates a spatial feature matrix based upon the plurality of subimages. 

14. The vehicle occupant classification system of claim 13 further including at least 
one filter generating the spatial feature matrix based upon the plurality of subimages. 

15. The vehicle occupant classification system of claim 14 further including an image 
processor for altering the image based upon lighting conditions and based upon motion. 

16. The vehicle occupant classification system of claim 15 wherein the processor 
analyzes the spatial feature matrix to determine the occupant classification using a neural 
network. 

17. The vehicle occupant classification system of claim 10 further including a 
temporal smoothing filter applying a decaying weighting function to a plurality of previous 
occupant classifications to determine a present occupant classification. 

18. The vehicle occupant classification system of claim 17 further including a 
confidence weighting function applied to the plurality of previous occupant classifications to 
determine the present occupant classification. 

19. The vehicle occupant classification system of claim 10 further including a 
plurality of digital filters extracting low-level descriptors from each of the subimages, the 
processor analyzing the low-level descriptors to determine the classification of the occupant. 
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20. A method for classifying an occupant including the steps of: 

a) capturing an image of a plurality of occupant areas; 

b) dividing the image into a plurality of subimages of predetermined spatial regions; 

c) generating a plurality of low-level descriptors from each of the plurality of subimages; 

d) analyzing the low-level descriptors; and 

e) classifying an occupant in each of the plurality of occupant areas based upon step d). 

21 . The method of claim 20 wherein said step d) further includes the step of analyzing 
the low-level descriptors based upon a set of training data. 

22. The method of claim 21 further including the step of creating the set of training 
data by capturing a plurality of images of known occupant classifications of the occupant area. 

23. The method of claim 20 wherein said steps d) and e) are performed using a neural 
network. 

24. The method of claim 20 wherein said step d) is based upon system parameters 
including an orientation or a location from which the image is captured relative to the occupant 
area. 
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EVIDENCE APPENDIX 



None. 
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RELATED PROCEEDINGS APPENDIX 



None. 
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